fix: dbt tool regression — schema format mismatch, silent failures, wrong results#263
Merged
anandgupta42 merged 7 commits intomainfrom Mar 18, 2026
Merged
fix: dbt tool regression — schema format mismatch, silent failures, wrong results#263anandgupta42 merged 7 commits intomainfrom
anandgupta42 merged 7 commits intomainfrom
Conversation
…rong results (#261) Root cause: `schema-resolver.ts` passed flat schema_context (`{"table": {"col": "TYPE"}}`) directly to `Schema.fromJson()`, which expects the Rust `SchemaDefinition` format (`{"tables": {"table": {"columns": [{"name": "col", "type": "TYPE"}]}}}`). The parse silently failed, falling back to an empty schema, causing all schema-dependent tools to produce empty/wrong results. Fixes: - `schema-resolver.ts`: Auto-detect and normalize 3 schema_context variants (flat map, array-of-columns, SchemaDefinition) before calling `Schema.fromJson()` - `sql-analyze.ts`: Check `result.error` instead of `!result.success` to distinguish "issues found" from "analysis failed" - `altimate-core-grade.ts`: Map Rust field names (`overall_grade`, `scores.overall`) instead of `grade`/`score` - `altimate-core-complete.ts`: Use `data.items` (Rust field) with fallback to `data.suggestions` - `altimate-core-schema-diff.ts`: Use `has_breaking_changes` (Rust) and format tagged enum changes correctly - `altimate-core-rewrite.ts`: Use `data.suggestions` (Rust) with fallback to `data.rewrites` - `altimate-core-fix.ts`: Map `fixes_applied`/`unfixable_errors` (Rust fields) instead of `changes`/`errors` - `altimate-core-column-lineage.ts`: Show `column_dict` and use `lens_type` (Rust) for transform info - `sql/register.ts`: Populate `can_auto_apply`, `original_fragment`, `rewritten_fragment` in `sql.rewrite` handler Closes #261 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Full simulation of 77 tests across all altimate-core tools revealed
6 more field name mismatches between Rust napi output and TS formatters:
- `altimate_core.transpile`: QUALIFY post-processing checked `data.sql`
but Rust returns `transpiled_sql` (array)
- `altimate_core.resolve_term`: Rust returns array, `toData()` converts
to `{'0': ...}` object — now wraps as `{ matches: [...] }`
- `altimate_core_classify_pii`: checked `data.findings` but Rust returns
`columns` with `classification`/`suggested_masking` fields
- `altimate_core_query_pii`: checked `data.exposures` but Rust returns
`pii_columns` with `risk_level`/`suggested_alternatives`
- `altimate_core_testgen`: test cases have `inputs`/`expected` fields,
not `sql` — formatter now shows input values and expectations
- `altimate_core_prune_schema`: checked `data.pruned` but Rust returns
`pruned_schema_yaml`/`relevant_tables`/`tables_pruned`
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Two new test files exercising all 36 altimate-core napi functions through the dispatcher layer: - `issue-261-e2e.test.ts`: 14 tests reproducing all 11 broken tools from #261 plus schema format variant tests - `edge-cases-simulation.test.ts`: 76 tests covering empty inputs, complex SQL patterns (nested subqueries, CTEs, window functions, correlated subqueries), column lineage through transformations, schema diff with large schemas, PII detection, multi-dialect transpilation, DDL roundtrip, equivalence checks, policy enforcement, and migration safety analysis Total: 192 tests, 404 assertions, 0 failures. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Stress simulation covering: SQL pattern variants (recursive CTEs, LATERAL joins, GROUPING SETS), lint anti-pattern detection, column lineage through 10 transformation types, schema diff mutation matrix (100-column tables, special chars), 10-dialect transpile matrix, grading consistency, fuzzy fix matching, testgen feature coverage, cursor-position completions, PII detection, and term resolution. Also fixes schema-resolver to skip tables with empty column lists (Rust engine requires >= 1 column per table). Total simulation suite: 288 tests, 867 assertions, 0 failures. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Moved all simulation tests into proper e2e test files following the project convention (`*-e2e.test.ts` in `test/altimate/`): - `altimate-core-e2e.test.ts` — 100 tests: all dispatcher methods, schema format variants, error recovery, complex SQL patterns - `altimate-core-stress-e2e.test.ts` — 80 tests: dialect matrix, fuzzy fix matching, grading consistency, testgen coverage, cursor completions, PII detection, lineage tracing, schema diffs - `issue-261-e2e.test.ts` — 14 tests: regression tests for #261 Removed standalone simulation files (full-simulation, edge-cases, stress-simulation) — content consolidated into the e2e files above. Total e2e suite: 180 tests, 539 assertions, 0 failures. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… PII field fallbacks 1. `isSchemaDefinitionFormat`: now validates that values under `tables` contain a `columns` array, preventing false positives when a flat schema has a table literally named "tables" 2. `formatQueryPii` / `formatClassifyPii`: added fallbacks for missing `table`/`column` properties to prevent "undefined.column_name" output 3. Added test for the "tables" false-positive scenario Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
| lines.push("\nChanges applied:") | ||
| for (const c of data.changes) { | ||
| lines.push(` - ${c.description ?? c}`) | ||
| for (const c of fixes) { |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Contributor
Author
There was a problem hiding this comment.
Fixed in commit 0bf5bedc8. Changed the condition from data.fixed && data.fixed_sql to data.fixed_sql && data.fixed !== false — so a valid fixed_sql is shown when fixed is either true or absent (undefined). Only an explicit fixed: false suppresses the output.
1. `formatFix`: changed condition from `data.fixed && data.fixed_sql` to `data.fixed_sql && data.fixed !== false` so a valid `fixed_sql` is not silently discarded when `fixed` field is absent 2. `sql/register.ts`: wrapped all 10 composite SQL handlers in exportable `registerAllSql()` function (mirrors `registerAll()` pattern from `altimate-core.ts`) 3. All 3 e2e test files now call both `core.registerAll()` and `sql.registerAllSql()` in `beforeAll` to survive `Dispatcher.reset()` from other test files — fixes CI "No native handler" failures Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes 11 broken/degraded dbt tools reported in #261 by addressing two root causes:
Schema format mismatch (root cause for 9 tools):
schema-resolver.tspassed flatschema_context({"table": {"col": "TYPE"}}) directly toSchema.fromJson(), which expects the RustSchemaDefinitionformat ({"tables": {"table": {"columns": [{"name": "col", "type": "TYPE"}]}}}). The parse silently failed, falling back to an empty schema.Field name mismatches (6 tools): Tool formatters referenced wrong field names from the Rust napi output (e.g.,
gradevsoverall_grade,suggestionsvsrewrites,has_breakingvshas_breaking_changes).Tools fixed:
sql_rewrite/lineage_check— schema parse errors → now workaltimate_core_schema_diff— "Schemas are identical" bug → now detects changessql_analyze— "Unknown error" → now shows found issuesaltimate_core_grade— "Grade: undefined" → now shows A-F gradealtimate_core_fix/altimate_core_rewrite— empty results → now work with schemaaltimate_core_column_lineage/altimate_core_testgen/altimate_core_complete— empty results → now work with schemaType of change
Issue for this PR
Closes #261
How did you verify your code works?
schema-resolver.test.ts)tool-formatters.test.ts)altimate-core-native.test.ts)biome checkclean on all modified filesturbo typecheckpassesChecklist
🤖 Generated with Claude Code